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09/224,583 



Examiner 

Michael N. Opsasnick 



Applicant(s) 

BEUTNAGEL ET AL 



Art Unit 

2655 



The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). tn no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )I3 Responsive to communication(s) filed on October 16,2003 . 
2a)l3 This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Qt/ay/e, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) g] Claim(s) 1.2.12-34 and 36-70 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1-28.31-34.36-40.43-51 and 57-65 is/are rejected, 

7) K! Claimfs) 20-25.27-30.41.42.52-56 and 66-70 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) Q The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

1 1) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)D All b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attachment(s) 

1 ) □ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) Paper No(s). . 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-152) 

3) □ Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other: 
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DETAILED ACTION 



Claim Rejections - 35 USC § 112 



1 . The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

2. Claim 20 contains narrative and idiomatic language, in particular the phrase "over and 
above", which renders the claim indefinite. Examiner suggests using other phraseology to 
distinguish the separate video synthesis command signals (for example, \ ...second converter for 
generating a second set of video synthesis command signals, wherein said second set of video 
synthesis command signals. . ..'). Clarification is required. 



3. Claims 29,30, 41,42,52-56, and 66-70 are objected to as being dependent upon a rejected 
base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

4. Claims 20,21-25,27,28 are objected to as being dependent upon a rejected base claim, but 
would be allowable if, a) claim 20 is rewritten to overcome the 1 12 second rejection above, and 
b) rewritten in independent form including all of the limitations of the base claim and any 
intervening claims. 



Allowable Subject Matter 
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Claim Rejections - 35 USC § 103 



5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 1,2,12-19,26,31-34,36-40,43-51,57-65 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Applicant's Admitted Prior Art in view of Bothe (Audio to Audio-Video 
Speech Conversion with the Help of Phonetic Knowledge Integration (IEEE, Jan. 1997) . 

Applicant's Admitted Prior Art (the prior art sections of applicant's spec, pp. 1-3, 
pertaining to the MPEG 4 standard) teaches a TTS stream input into a decoder, thru a 
synthesizer, and into a compositor (see Fig. 1 of applicant's specification). Furthermore, 
applicant's admitted prior art teaches FAPS and a FACE model into a FRM to be input to 
the compositor (Fig. 1). Applicant's admitted prior art also teaches the definition of the 
FAP's (pp 2 of applicant's specification), including 10 groups of parameters (ranging 
from different parts of the face, along with viseme information). The information with 
respect to the FAP's is used along with decoded audio information, to produce an 
animated video synthesized with the audio (applicant's admitted prior art, pp. 3, lines 22- 



26). 
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{As a side note, throughout applicant's admitted prior art description on pps. 1-6 
of the specification, applicant stresses that the novelty of the instant invention is the use 
of phonemes, derived from the "tts stream decoder" output, generating FAPS to be input 
into the FRM, for eventual synthesis (applicant's specification, pp 2 lines 3-5, pp 4 lines 
10-11, and pp. 5 lines 3-5; and pp 6 lines 1-6).} 

To summarize, applicant's admitted prior art teaches a systems with a TTS stream 
into a decoder, synthesizer, and compositor, along with a face model and FAPS into the 
compositor such that the output of the compositor is a synthesized audio/visual, wherein 
the timing of the audio and visual information is derived from the streams (applicant's 
admitted prior art, pp 3 lines22-28). 

Applicant's admitted prior art does not explicitly teach the use of phonemes, 
derived from the decoded output of the TTS stream, to provide more FAP information 

into the FRM (into the COMPOSITOR) for synthesis. Bothe (Audio to \ however, 

teaches the derivation of phonemes from a TTS stream (using a transform) to influence 
the animation of the phoneme (Fig. 1, pp 1632, second column; pp 1635, second column 
- page 1637, end of second column before 'Conclusion and Future Work). Therefore, it 
would have been obvious to one of ordinary skill in the art of speech animation to modify 
the teachings of Applicant's Admitted Prior Art, as explained above, with phoneme based 
animation information because it would advantageously produce an illusion of a mouth 
moving to generate the sounds heard by synchronizing audio playback of a phoneme with 
display of a mouth shape corresponding to the phoneme, for application such as video 
telephones and lipreading training ( Bothe (Audio to ) , abstract). 
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As per claims 1 and 2,12,14,16,31,33 the combination of Applicant's Admitted 

Prior Art in view of Bothe (Audio to ) teaches a decoder responsive to an input signal 

comprising text and FAP information (applicant's admitted prior art), generating addition 
FAP information from phoneme information to add to the FAP information (applicant's 

' Bothe (Audio to V admitted prior art, as demonstrated above), and a face rendering 

module responsive to the face model and decoder output (applicant's admitted prior art) 

As per claims 17,23,36, the combination of Applicant's Admitted Prior Art in 

view of Bothe (Audio to ) teaches FAP based command signals (applicant's admitted 

prior art) 

As per claims 34,37 the combination of Applicant's Admitted Prior Art in view of 

Bothe (Audio to ) teaches prosody and timing information (Bothe (Audio to \ pp 

1635, bottom paragraph to pp 1636, bottom) 

As per claims 2,24,25, the combination of Applicant's Admitted Prior Art in 

view of Bothe (Audio to ) teaches basic FAPS (applicant's admitted prior art - non- 

viseme based, and other grouping of FAPS). 

As per claims 13,15,18,19,32, the combination of Applicant's Admitted Prior Art 

in view of Bothe (Audio to ) teaches sound, encoded voice, sound synthesizer, text, 

video synthesizer, and face model information (applicant's admitted prior art). 

As per claims 26-28, 38-40, the combination of Applicant's Admitted Prior Art in 
view of Bothe (Audio to ) teaches an interpolated signal including phoneme, timing, 
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and command information ( Bothe (Audio to ) data structure, Fig. 12, the neural 

networked derived information, as applied to applicant's admitted prior art structure) 

As per claims 16,20-22,43 and 57, the combination of Applicant's admitted prior 

art in view of Bothe (Audio to ) teaches receiving an input that includes text 

specification, FAP information, output synthesized voice (applicant's admitted prior art), 
phoneme output with addition information (applicant's admitted prior art modified by 

Bothe (Audio to ) teachings, Bothe (Audio to ) , Fig. 1, as pointed to above), 

generating a sequence of facial animation parameters from signals of said second output 

(applicant's ' Bothe (Audio to V admitted prior art), and rendering images from the 

output signal and combining the synthesized voice and images (applicant's fig. 1, FRM, 
synthesizer, and compositor). 

As per claims 44 and 58, applicant's admitted prior art teaches receiving the 
input from a remote locations via a network 

As per claims 45 and 59, the combination of Applicant's Admitted Prior Art in 

view of Bothe (Audio to ) teaches bookmarks marking phoneme boundaries (Fig. 12). 

As per claims 46 and 60, the combination of Applicant's Admitted Prior Art in 

view of Bothe (Audio to ) teaches information about the FAP's (applicant's admitted prior art 

of groupings of FAP's). 

As per claims 47 and 61, the combination of Applicant's Admitted Prior Art in 

view of Bothe (Audio to ) teaches amplitude information (applicant's admitted prior art, pp2 

lines 10-13 of applicant's specification) 
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As per claims 48 and 62, the combination of Applicant's Admitted Prior Art in 

view of Bothe (Audio to ) teaches duration ( Bothe (Audio to \ pp 1636, first column, 

frame length) 



Art in view of Bothe (Audio to ) teaches a specified transition path, chosen by the animation 

module based on phoneme information via the bookmark (pp 1636, the computer animation 
based on the phoneme input sequence and key vectors and the MLP). 



7. Applicant's arguments filed 10/16/2003 have been fully considered but they are not 
persuasive. As per the arguments on page 12 of the response (first three paragraphs), examiner 
responds that applicant is arguing the newly amended claim language ^please see the rejections 
given above. As per applicant's arguments pertaining to a TTS stream on page 12, examiner 
notes that the claim language pertains to text and FAP information, not a TTS stream. As per 
applicant's comments with respect to the combination of applicant's admitted prior art and the 
Bothe reference (on the bottom of page 12 to top of page 13 of the response), examiner argues 
that the statements made by the applicant's representative is pure conjecture and speculation, 

especially pertaining to 'No skilled artisan Furthermore, examiner points to the stated 

reason for combining the two references as shown above. As per applicant's arguments on page 
13 of the response with respect to commingling, examiner points to the referred passage of 
applicant's admitted prior art (page 5, lines 3-6), teaching co-mingled information of the input 



As per claims 49-51 and 63-65, the combination of A pplicant's Admitted Prior 



Response to Arguments 



• 
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stream. As per applicant's arguments on top of page 14 of the response with respect to non- 
viseme FAP's, examiner points to applicant's admitted prior art teaching non-viseme FAPS 
(page 2 of applicant's specification, lines 28-30). As per applicant's arguments in the middle of 
page 14 of the response, examiner argues that a detailed explanation of the combination of prior 
art on pages 3-4 of the previous office action provides a clear base of applicant's admitted prior 
art referrals as applied to the claim language. As per applicant's arguments with respect to audio 
and vid^signals, examiner argues that applicant's admitted prior art (the MPEG 4 standard), has 
both audio and video system signals. As per applicant's arguments on page 16 with respect to 
interpolation, examiner argues that a second definition of interpolating (by Riverside Webster's 
II dictionary) is 'to determine a value of a function between known values by a procedure or 
algorithm different from that specified function itself, which is taught by the referred passage of 
Bothe. As per the similar arguments presented on page 16 to page 19 of the response 
corresponding to the addressed arguments on page 12-16 of the response, examiner offers 
likewise corresponding rebuttals to these arguments presented above. 



8. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 



Conclusion 
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will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

9. Any response to this action should be mailed to: 



Commissioner of Patents and Trademarks 
Washington, D.C. 20231 
or faxed to: 
(703) 872 9314, 

(for informal or draft communications, please label "PROPOSED" or "DRAFT") 
Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal 
Drive, Arlington. VA., Sixth Floor (Receptionist). 



10. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michael Opsasnick, telephone number (703)305-4089, who is 
available Tuesday-Thursday, 9AM-4PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ms. Doris To, can be reached at (703)305-4827. The facsimile phone number for 
this group is (703)872-9314. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the Group 2600 receptionist whose telephone number is (703) 305-4750, 
the 2600 Customer Service telephone number is (703) 306-0377. 



mno 



1/9/2004 




